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Introduction 

Training employees is integral to many organizations. When training attrition 
occurs, it is not unusual for managers to ask human resources specialists to conduct 
validation studies and set higher standards for selection. This may help in many cases, but 
sometimes is not fruitful. The following is an example of a training attrition situation 
where a valid predictor exists, is in place, but cannot be expected to reduce attrition. 
Other factors such as motivation and job design must be considered. 

The job of weapons director (WD), once only for commissioned officers, has 
recently become available to enlisted U. S. Air Force (USAF) members. Currently, WD 
teams can be made up of both officers and enlisted members. The goal is to have one 
officer as a supervisor for about five enlisted members. 

All enlisted members must be in their second four-year term of enlistment. This 
means that they have already been trained in another USAF technical specialty. Soon 
after enlisted WD training began, attrition problems were observed. Across the USAF 
typical attrition in enlisted technical training is about 5%. In the enlisted WD training it 
has run about 9% for several years, but 20% for the most recent year (1996). Because 
selection was by appointment or voluntary and because no test score minimums existed, 
senior managers sought a solution in the setting of standards for a valid selection 
composite. This effort is reported in the current study. 

Method 


Participants 


The participants were 353 enlisted Air Force members who attended WD training. 
All were in their second enlistment tour. They had successfully completed technical 
training for and perfonned on the job in their prior technical specialty. Of these, 321 

1 Previously published as Carretta, T. R., & Ree, M. J. (1997). Lack of ability is not always the reason for 
high attrition. Proceedings of the 39th Annual Conference of the International Military’ Testing 
Association, Sydney, Australia, 71-75. 
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successfully completed WD training and 32 attrited from training. All enlisted WD data 
were collected between 1990 and 1996. 

They were selected for USAF enlistment, at least in part, by scores on the Anned 
Services Vocational Aptitude Battery (ASVAB; Ree & Carretta, 1994). Due to this 
selection, they constituted a range restricted sample. There is no official screening process 
for enlisted WD training and almost all trainees were non-volunteers. 

Measures 

Predictors. The ASVAB is comprised of 10 tests that measure general cognitive 
ability (g) and the lower-order factors of verbal/math, speed, and technical knowledge 
(Ree & Carretta, 1994). The 10 tests are General Science (GS), Arithmetic Reasoning 
(AR), Word Knowledge (WK), Paragraph Comprehension (PC), Numerical Operations 
(NO), Coding Speed (CS), Auto and Shop Information (A/S), Mathematical Knowledge 
(MK), Mechanical Comprehension (MC), and Electronics Information (El). 

The predictors were composites of the ASVAB tests. The USAF, like the other 
services, aggregates the tests into composites to create more reliable scores and to have a 
broad content for the composites. The Air Force composites include Mechanical (M = 
MC + GS + 2A/S), Administrative (A = NO + CS + WK + PC), General (G = WK + PC 
+ AR), and Electronic (E = AR + MK + El + GS). The composite scores are reported as 
normative percentiles. 

The literature suggests that prior educational achievement predicts future 
educational achievement. Therefore, two other predictors available for all participants 
were examined. The first of these was numbers of years of (non-USAF) schooling 
(EdYrs). The second was final school grade (PFSG) in previous USAF technical training. 

Criterion. The criterion for the regressions was the final school grade (FSG) earned 
at technical training. FSG was the proportion correct in a series of paper-and-pencil 
written tests. The maximum was 99 and the minimum passing score was 70. If a trainee 
earned grades below 70 they were dropped from training for academic reasons. 

During training there are a series of work samples called “progress checks” that 
involve realistic simulation of the job tasks. These are scored pass-fail. Trainees who fail 
progress checks several times can be recommended for elimination by instructors for non- 
academic (called “lack of progress”) reasons. A review of these recommendations is 
conducted by senior managers who either concur or retain the trainee. The training 
graduates are those who acquired the job knowledge tested in academic training and 
acquired the relevant job knowledge and skills tested by the progress checks. 
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Analyses 

Participants were characterized as having completed (n = 321) or attrited (n = 32) 
from training. Among those who attrited from training, two reasons could be detennined; 
academic failures and other failures. Mean scores of those who completed and failed to 
complete training were computed for the ASVAB composites, years of education, and 
prior technical training final school grade. 

Those participants who attrited from enlisted WD training had either no or 
incomplete criterion data and could not be included in regression analyses. For the 
participants who completed technical training, regressions of the FSG on the four 
ASVAB composites, years of education, and PFSG were computed. The Type I error 
rate was specified as p < .01. After statistical significance testing, these data were 
corrected for range restriction (Lawley, 1943) and the same regressions again computed. 

To correct for the upward-bias in R, we applied Stein’s Operator (Kennedy, 
1988; Stein, 1960) to estimate shrinkage in cross-application. Stein’s Operator estimates 
the mean R coming from all possible cross-applications of the regression coefficients in 
samples of the size upon which the coefficients were estimated. It takes into account the 
sample size, number of predictors, and the estimated multiple correlation. 

Finally, minimum cut scores were established with reference to dual goals. The 
first goal was to minimize attrition in training and the second goal was to reduce false 
positives. The goal of minimizing training attrition was investigated by inspecting 
predicted scores in training from the regression equation and finding the value below 
which training grades are expected to be less than 70. Reduction of false positives was 
investigated by means of two-way distributions of passing/failing and a selector 
composite. 


Results and Discussion 

As would be expected for a sample of participants subjected to prior selection, the 
standard deviations of the composites were reduced from the normative sample. For M, 
A, G, and E the standard deviations of the composites were 67%, 59%, 47%, and 53% of 
their respective nonnative values. Table 1 shows the means, standard deviations, and 
conelations of all variables before and after correction for range restriction. 

The simple bivariate conelation of each composite with the FSG criterion can be 
found in Table 1. All of these correlations were statistically significant. Because the 
corrected conelations are superior statistical estimates we shall limit our discussion to 
them. The G composite (r = .58) was most predictive followed closely by the E 
composite ( r = .56). The M composite was less predictive (r = .43) as was the A 
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composite (r = .40). EdYrs (r = .35) and PFSG (r = .49) were also less predictive than 
either G or E. 
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Table 1. Means, Standard Deviations, and Correlations for the Variables 



M 

A 

G 

E 

EdYrs 

PFSG 

FSG 

Mean 

SD 

M 

1.00 

.13 

.43 

.64 

.07 

.19 

.15 

71.52 

19.35 

A 

.47 

1.00 

.43 

.35 

.06 

.14 

.08 

70.88 

18.60 

G 

.71 

.77 

1.00 

.70 

.14 

.29 

.31 

73.47 

13.59 

E 

.82 

.70 

.93 

1.00 

.15 

.23 

.25 

71.66 

15.18 

EdYrs 

.24 

.25 

.33 

.33 

1.00 

.18 

.23 

13.02 

.82 

PFSG 

.45 

.44 

.56 

.54 

.30 

1.00 

.31 

87.80 

6.01 

FSG 

.43 

.40 

.58 

.56 

.35 

.49 

1.00 

88.81 

4.72 

Mean 

50.33 

51.41 

51.00 

50.33 

12.80 

84.61 

86.36 



SD 

28.81 

31.36 

29.19 

28.83 

.85 

6.97 

5.50 




Note. Values above the diagonal are observed data and those below have been corrected 
for range restriction. 


FSG was regressed on each composite, EdYrs, and PFSG. Table 2 shows the 
multiple correlations from the regressions. Again, the E and G composites were preferable 
to either the M or A composites as they were more valid. Because the multiple 
correlation is an upwardly-biased estimator, it is necessary to reduce the R as would 
occur in cross-application of the regression weights in another sample. Stein’s Operator 
(Kennedy, 1988; Stein, 1960) was applied and the resultant shrunken Rs are in the last 
column of Table 2. These values are those that would be expected when the regression 
equation was validated on a separate set of applicants. 

Table 2. Regression Results 


Predictors 

R 

R c 

R s 

Regression Equations 

M, EdYrs, PFSG 

.37 

.57 

.55 

46.44 + .04M + 1.23EdYrs + .25PFSG 

A, EdYrs, PFSG 

.36 

.56 

.54 

45.60 + .03A + 1.24EdYrs + .27PFSG 

G, EdYrs, PFSG 

.42 

.63 

.61 

55.72 + .07G + .95EdYrs + . 17PFSG 

E, EdYrs, PFSG 

.40 

.61 

.59 

54.08+ .07E + .97EdYrs + .19PFSG 


Note. R is the uncorrected observed multiple correlation, R c is the corrected multiple correlation. R s is R £ 
shrunken to estimate cross-validated values. 

The incremental validity of adding the additional infonnation in the EdYrs and PFSG 
variables was rather small. The practical effect was an approximately .03 increase in R. 
Further, the choice between E and G cannot be made on predictive efficiency grounds 
alone. Practical considerations are also important. 
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The distributions of the E composite for women and men differ. At the nonnative 50th 
percentile, 59% of the men qualify but only 42% of the women qualify. For the G 
composite the same values are 54% and 47%, a much smaller difference. Given that the E 
and G composites have similar validities it would minimize adverse impact to use the G 
composite. 

After identifying the prefened composite it is necessary to establish minimum cut scores 
for operational use. The cut score should address the goals of the selecting agency, such as 
minimizing false positives, while being mindful of the cost of recruiting. By evaluating the 
predicted scores from the regression equation we can estimate the expected training grade 
performance of applicants. For the G composite normative percentiles of 20, 50, and 80 
the predicted training grades are: 83, 86, and 90. False positives were found to occur for 
the G composite score ranging rather uniformly from 34 to 86 with one case at 99. Fully 
half the failures have G composite scores of 70 or better. This suggests that attrition was 
not caused by a lack of cognitive ability. At this score level, 70, the attrition would be 
5%, the USAF average. However, the cost of recruiting and demand for high scoring 
ainnen is great. 

We first looked at the scores of the participants who were classified as attritees from 
training. Of the 32, only three were academic failures. Academic failure results when a 
student does not achieve sufficiently high scores on the course tests. The causes of 
attrition for the other 29 was a failure to progress in training. These participants 
performed poorly on the work sample progress checks. The mean score on the G 
composite for the three academic failures was 55. The predicted training grade for G = 55 
was about 87. These three academic failures should have completed training. Their 
success in prior technical training and success on the prior job also suggests that they 
should have completed this training. 

For the 29 non-academic failures the mean G score was 67. For the graduates the mean G 
score was 73, not much above the non-academic attritees. The distribution of G scores for 
the graduates ranged from 39 to 99, almost identical (34 to 99) to the distribution of the G 
scores for the non-academic attritees. In fact, the distributions are very similar with only 
small differences in shape and cumulative proportions. There were no notable differences 
in ability between those who successfully completed training and those who failed to 
complete training for non-academic reasons. 

We speculate that the non-academic attritees failed to complete training for what might be 
termed “lack of motivation.” First, almost all were non-volunteers. This may have had a 
demotivating effect. Second, the job characteristics may be demotivating. For example, 
extensive travel up to 220 days per year is required. Further, unlike fonner officer 
weapons directors trainees who left the Air Force if they failed training, enlisted trainees 
can return to other military occupations. Some trainees may have deliberately perfonned 
poorly in the progress checks to be released from training. With an average G composite 
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of 73 for all accessions, it is clear that this job has high ability trainees. It may be that this 
is not an attractive job as it is currently structured and managed. 

There are several corrective paths that might be explored. The utility of non-cognitive 
measures could be examined for predictiveness. Conscientiousness, a personality variable 
that addresses motivation, has been shown to be predictive of job performance across a 
wide variety of occupations. Barrick and Mount, (1991) conducted a meta-analysis of the 
predictiveness of several personality variables for many job performance criteria. These 
criteria included training perfonnance, job perfonnance, tenure and salary. Barrick and 
Mount estimated the mean true validity of conscientiousness as .22 with little variability 
due to job type such as professional, blue collar, or sales. 

However, it is unlikely that the non-academic attritees lacked in conscientiousness. They 
have been through basic military training, Air Force technical training, and completed one 
four-year term of enlistment. 

If some characteristics of the job were found to be demotivating, job redesign could be 
undertaken to change onerous aspects such as travel requirement or long hours. Campion 
(1988) has developed a self-report instrument for job redesign that covers four areas: 
Motivation, Mechanical, Biological, and Perceptual/Motor. Campion’s instrument could 
be used to assess and recommend changes to job design. 

The impact of bonuses or other special incentives could be estimated. Finally, these three 
changes (non-cognitive measures, job redesign, incentives) might be used in some 
combination. 

Although managers frequently perceive that raising standards will solve training attrition 
problems, this is not always the case. In the situation reviewed here there was a valid 
predictor available, trainees were highly selected on ability (and probably 
conscientiousness), but attrition from training was still high. In a case such as this, raising 
selection standards for ability cannot be expected to reduce attrition. The problem 
appears to be in motivation or in job design. To provide answers for mangers these other 
aspects, motivation and job design must be evaluated and appropriate remediation 
applied. 
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